Apparent requirement for nested includes and/or simpler markdown inclusion

[SteveALee]

Background

I'm working on the W3C note Content Usable in order to also render the Design Guide section as a set of web pages in the W3C WAI website.

The WAI website is built using the Jekyll SSG and the design guide section is currently a single HTML + respec file. The basic goal is to refactor the Design Guide into a number of files that can be consumed by the SSG with minimal processing. The structure will also be processed by respec and saved as usual for the W3C note.

Is your feature request related to a problem? Please describe.

Matching includes to section structure

So the structure of the Design guide is as follows:

• heading Design Guide
• section intro
  • heading Objective 1
  • section intro
    • heading Pattern 1
    • section 1 intro
    • section 2
    • heading Pattern 2
    • heading Objective 2
    • section

With each heading and section(s) being contents of one file.

file DG:

• heading Design Guide
• section intro
• include file O_1

file O_1

• heading Objective 1
• section intro
• include file P_1

File P_1:

• heading Pattern 1
• section 1 intro
• section 2

etc

This would be naturally suggest nested includes but they are not supported.

Flattening the structure requires markup or content that should be in the included file to be in the including file. Plus I found it impossible to get the section tags correctly nested. This all complicates file processing and a 1 to 1, section to file mapping appears to be the natural solution.

Markdown

Ideally the files will all contain markdown but as the content is currently HTML that will be OK.

Describe the solution you'd like

• Nested includes so files map 1 to 1 to sections
• top level just a list of includes - don't mind what tag - div would do, especially if auto replace is used.
• Each file can start with a Heading and I don't mind if the section tag gets added automatically as part of the include (as it seems to right now) or we have to add them to the content (but not if markdown).

And also when using markdown:

• markup declared at the point of inclusion - ie data-attrib on the include

Describe alternatives you've considered

• Flattening the includes and playing around with missing/extra section end tags to get the correct structure.
• tried markdown but i could get the include markdown to only work in limited circumstances when working throughthe permutations. That might well me my lack of understanding.

Any suggestions are most welcome on how to implement this now, or with some tweaks to respec for nested include of html and markdown.

PS @michael-n-cooper is also currently looking at the refactoring of the source files as part of the note release process.

[sidvishnoi]

I'm hesitant to support nested includes, as it makes it difficult for the tools (e.g. pr-preview) that need to parse the data-include to work correctly, and they're are also slow. I think what you're trying to achieve can be accomplished with flattened includes. Let me try to address the problems you're facing, and see if ReSpec really needs nested includes..

also render the Design Guide section as a set of web pages in the W3C WAI website. [...] The basic goal is to refactor the Design Guide into a number of files that can be consumed by the SSG with minimal processing

I'm assuming this would require adding some Jekyll specific front-matter to all the source files also? Currently, ReSpec doesn't parse/remove the front-matter.

Plus I found it impossible to get the section tags correctly nested.

Perhaps this file (source of https://respec.org/docs/) could be of some help? It makes extensive use of flattened data-include (which are markdown pages from the wiki), and is able to achieve arbitrary nesting as needed.

I can write a minimal example to show headings working correctly if that would be helpful.
Edit: https://boring-nobel-77ed93.netlify.app/

Flattening the structure requires markup or content that should be in the included file to be in the including file.

Flattening the includes definitely does mean repeating includes in each page, but it makes sense as they're separate ReSpec documents.

[SteveALee]

Thanks @sidvishnoi .

'm hesitant to support nested includes, as it makes it difficult for the tools (e.g. pr-preview)

There are tools other than respec that process respec enhanced HTML source docs? I hadn't considered that as something that would happen. But surely use of nested includes is optional and if those other tools are not be used on a document then it should be reasonable to allow nested includes. Perhaps with a warning in the docs?

I'm assuming this would require adding some Jekyll specific front-matter to all the source files also? Currently, ReSpec doesn't parse/remove the front-matter.

Actually was thinking of the reverse. parsing the individual source files to generate either YAML frontmatter or perhaps a Jekyll data file. The latter looks possible even though Jekyll is not as flexible with data as 11ty. Some of the data depends on structure so may be fiddly to generate (my next task is to spike this). an alternative would be to use filename convention to specify the structure.

As an alternative, allowing Yaml frontmatter in a respec source doc would be a most welcome addition though. :)

Thank you so much for the examples!

In this case there will be a single top level "Design Guide" doc with all the includes.

Flattening the includes definitely does mean repeating includes in each page

That's not a problem as there is a single top level document (content usable). It's unfortunate that that already includes the document I want to structure so I have to push all the includes up, so increasing the complexity of the top document and how it is parsed to get at the Design Guide section contents

[sidvishnoi]

There are tools other than respec that process respec enhanced HTML source docs?

pr-preview is one of them. In future, ReSpec's Node.js API might also make use of parsing data-include. spec-prod will be parsing data-include from source also.

But surely use of nested includes is optional and if those other tools are not be used on a document then it should be reasonable to allow nested includes. Perhaps with a warning in the docs?

Partial support sounds reasonable to me. cc: @marcoscaceres

It's unfortunate that that already includes the document I want to structure so I have to push all the includes up

Sorry if I misunderstood, does this mean nested includes won't be helpful?

[SteveALee]

It's unfortunate that that already includes the document I want to structure so I have to push all the includes up

Sorry if I misunderstood, does this mean nested includes won't be helpful?

No, quite the opposite :) nested includes let's us place all the design guide includes in the design guide file.

Otherwise every include has to be in the top level doc which will make it messy and harder to extract metadata for the design guide frontmatter.

[SteveALee]

@sidvishnoi

Edit: https://boring-nobel-77ed93.netlify.app/

Thanks for that. I can't figure how to view the source code for it though. I tried saving the file but it appears to be already processed.

thanks

[sidvishnoi]

@SteveALee You can right-click > "view page source" on any page to get unprocessed source code:
view-source:https://boring-nobel-77ed93.netlify.app/objective-1.html

[SteveALee]

@sidvishnoi - Gosh, after all these years I never noticed that - always used "inspect element" or "Save As". Even works in FFx. \0/

Thanks for the education!!! :-P

[SteveALee]

@sidvishnoi Closing now as have solution to nesting thanks to you feedback. It seems the flexibility of replacing tag or not helps.

Markdown include is now working well for me too.

I put the frontmatter in a pre element which is removed by respec. Pre should stop editors messing up the format. No need for respec support of frontmatter

FYI - here's were we have got to so far. Uses a simple bash script to convert included files to Jekyll pages for the WAI website. All works very well

[sidvishnoi]

Happy to know it worked for you @SteveALee!

Curious to know why using:

document.respec.ready.then(() => {
  postRespec();
});

instead of respecConfig.postProcess though.

[SteveALee]

I have no idea as I inherited it as is :) There also seem to be an onInclude handler that doesn't exist which I need to check.

I will pass the question on though.

thanks

[michael-n-cooper]

Steve asked me to answer the question 2 comments ago. In the past respecConfig.postProcess didn't work (or stopped working, or something). I got used to having to work around it, and assumed the documentation simply hadn't been updated (very super duper common problem). Further, the documentation for respecIsReady specifies it is the way to meet the use case. I assumed it was the feature replacing the postProcess directive. When respecIsReady stopped working, I had to dive into code to find it's been replaced by ready, so the documentation is clearly out of date on that one.

From the documentation, I have no idea what works, what is supposed to work but doesn't, and what is supposed to not work. I just do whatever I can find that gets me the result I'm looking for, and move on, I don't have time to figure out what is considered the "right" thing. Clearer, more comprehensive, and up-to-date documentation would certainly help.